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EXECUTIVE SUMMARY 



This report collects the results of all available empirical stud- 
ies on how vouchers affect academic achievement in public 
schools. Contrary to the widespread claim that vouchers hurt 
public schools, it finds that the empirical evidence consistently 
supports the conclusion that vouchers improve pubhc schools. 
No empirical study has ever found that vouchers had a nega- 
tive impact on pubhc schools. 

There are a variety of explanations for why vouchers might 
improve pubhc schools, the most important being that compe- 
tition from vouchers introduces healthy incentives for pubhc 
schools to improve. 

The report also considers several alternative explanations, be- 
sides the vouchers themselves, that might explain why pubhc 
schools improve where vouchers are offered to their students. 
It concludes that none of these alternatives is consistent with 
the available evidence. Where these claims have been directly 
tested, the evidence has not supported them. The only consis- 
tent explanation that accounts for ah the data is that vouchers 
improve pubhc schools. 

Key findings include: 

• A total of 17 empirical studies have examined how vouch- 
ers affect academic achievement in pubhc schools. Of these 
studies, 16 find that vouchers improved pubhc schools and 
one finds no visible impact. No empirical studies find that 
vouchers harm pubhc schools. 

• Vouchers can have a significant positive impact on pubhc 
schools without necessarily producing visible changes in the 
overaU performance of a large city’s schools. The overah 
performance of a large school system is subject to countless 
different influences, and only careful study using sound sci- 
entific methods can isolate the impact of vouchers from ah 



other factors so it can be accurately measured. Thus, the 
absence of dramatic “miracle” results in cities with voucher 
programs has no bearing on the question of whether vouch- 
ers have improved pubhc schools; only scientific analysis can 
answer that question. 

• Every empirical study ever conducted in Milwaukee, Flori- 
da, Ohio, Texas, Maine and Vermont finds that voucher pro- 
grams in those places improved pubhc schools. 

• The single study conducted in Washington D.C. is the only 
study that found no visible impact from vouchers. This is 
not surprising, since the D.C. voucher program is the only 
one designed to shield pubhc schools from the impact of 
competition. Thus, the D.C. study does not detract from 
the research consensus in favor of a positive effect from 
voucher competition. 

• Alternative explanations such as “stigma effect” and “re- 
gression to the mean” do not account for the positive effects 
identified in these studies. When these alternative explana- 
tions have been evaluated empiricahy, the evidence has not 
supported them. 



EMPIRICAL STUDIES FINDING THAT VOUCHERS... 





...improved public 
school outcomes 


...didn't visibly change 
public school outcomes 


...hurt public 
school outcomes 


MILWAUKEE 


5 


0 


0 


FLORIDA 


10 


0 


0 


OTHER PROGRAMS 


3 


0 


0 


WASHINGTON D.C. 


0 


1 


0 



Note: A total of 17 studies are represented here; figures do not sum to 17 because some studies include 

findings on more than one program. 
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A WIN-WIN SOLUTION: THE EMPIRICAL EVIDENCE ON HOW VOUCHERS AFFECT PUBLIC SCHOOLS 



INTRODUCTION 




This report reviews all the 
available empirical studies of how 
voucher programs affect academic 
achievement in public schools. 



School vouchers, which allow parents to use public funds 
to send their children to the school of their choice, public 
or private, are among the most prominent and successful 
reforms in the education field. Perhaps the single most im- 
portant question about vouchers is how they impact public 
schools. There are many people who agree that vouchers are 
good for the students who have the opportunity to use them, 
but are concerned about how vouchers impact the quality of 
education for other students who remain in public schools. 

Defenders of the government monopoly on schools frequent- 
ly claim that vouchers harm public schools. They claim that 
vouchers drain money and “cream” the best students. Voucher 
proponents, on the other hand, argue that vouchers improve 
public schools. They point to evidence that vouchers save mon- 
ey for public school budgets rather than “draining” money, and 
that vouchers do not only send the best students to private 
schools. The proponents argue that vouchers allow students to 
find the right schools to serve their individual needs, and intro- 
duce competition for students that creates healthy incentives 
that are lacking in the existing government school monopoly. 



A large body of empirical evidence speaks to this question. 
There are now 24 school choice programs in 14 states and 
Washington D.C. Over 160,000 students use these programs 
to attend private schools using public funds. The effects of 
these programs have been studied using scientific methods 
and are no longer the subject of mere speculation and anec- 
dotal observation. 



This report reviews all the available empirical studies of how 
voucher programs affect academic achievement in public 
schools. It also discusses the most important methodologi- 
cal issues confronted by research on this subject, including 
an explanation of why scientific studies are necessary for an 
accurate picture of whether vouchers are improving public 
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schools, and consideration of the alternative explanations 
for why public school improvements might be associated 
with voucher programs. 

Choice and Competition in Education 

Unfortunately, Americans are not accustomed to thinking of 
K-12 education in terms of choice. They expect and demand 
the right to select their own goods and services in everything 
from food, housing, clothing, transportation and medical care 
to magazines, haircuts, dry cleaning and video games. If gov- 
ernment attempted to assign people to live in certain neigh- 
borhoods or shop at certain grocery stores, they would howl 
in protest. Americans even expect and demand choice when 
it comes to education outside of K-12 schools— everywhere 
from colleges to trade schools to night classes. But when it 
comes to K-12 education, the idea that they would choose for 
themselves rather than having government dictate what they 
receive is new and sometimes uncomfortable. 

This helps to explain why many Americans readily accept 
claims about school vouchers that are empirically false or 
poorly reasoned. For example, when teachers’ unions claim 
that vouchers ‘‘drain money” from public schools, many 
Americans nod in agreement. But how would those same 
people respond if they were told that from now on they would 
have to receive all their medical care from a doctor assigned 
to them by the government, rather than from their current 
family doctor, on grounds that their choice to seek care from 
their current doctor “drains money” from the budget of the 
doctor chosen by the government? 

In fact, vouchers make public schools better off financially, 
rather than worse off. When students leave public schools 
using vouchers, not all the funding associated with those 
students goes with them. This means public schools are left 
with more money to serve the students who remain. State 



budgets also benefit because educating students in private 
schools rather than public schools not only accomplishes 
better results, it also costs less. From 1990 to 2006, the na- 
tion’s school choice programs saved a total of $422 million 
for local school districts and $22 million for state budgets.^ 

Similarly, the claim that vouchers “cream” the best students 
from public schools has no empirical evidence to support 
it. The best available analyses of this question have found 
voucher applicants to be very similar to the population of 
students eligible for vouchers in terms of demographics and 
educational background.^ In the Washington, D.C. voucher 
program, applicants were very similar to a representative 
sample of the eligible population citywide not only in terms 
of demographics, but also in their baseline test scores.^ 

Meanwhile, for similar reasons, the idea that vouchers might 
improve public schools seems counter-intuitive to many 
Americans. In fact, it is not hard to explain why vouchers 
would be expected to improve public schools. One reason is 
because vouchers allow parents to find the right particular 
school for each individual child. Every child is unique and 
has unique educational needs. 

But probably the most important reason vouchers would im- 
prove public schools is because they give parents a meaningful 
way to hold schools accountable for their performance. Under 
the current system, if a school isn’t doing a good job, the only 
ways to get a better school— purchase private schooling or 
move to a new neighborhood— are prohibitively expensive or 
cumbersome for many families. These options are especially 
difficult for low-income and disadvantaged students. 

Thus, in the absence of parental choice, schools lack the 
positive incentive for better performance that most other 
types of service institutions take for granted. Hospitals know 
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they must do a good job or else lose patients. Colleges must 
provide a good education (and other services and opportuni- 
ties that parents expect from colleges) or else lose students. 
Professionals like doctors and lawyers must provide good 
services or else lose clients. Stores must provide good value 
or else lose customers. 




This lack of connection between 
what Americans think about 
choice and competition in K-12 
education and what it thinks about 
choice and competition in virtually 
every other aspect of life is a 
great hindrance to accurate public 
discussion of school vouchers. 



With vouchers, those positive incentives we take for granted 
everywhere else are provided for schools. If a public school 
is providing adequate services, parents can leave their chil- 
dren where they are and be no worse off. But if not, parents 
can choose a private school that will serve their children 
better. Either way, schools know that parents have the power 
to hold them accountable. 

The same Americans who have difficulty with the idea that 
competition improves schools have no difficulty applying the 
same concept everywhere else. They know that monopolies 
provide poor quality because they have little incentive to 
serve their clients well. And when they get bad service, they 
say, ‘TTl take my business elsewhere” because they know 
that this provides an incentive for better service. 

They do this even in fields like medical care where the ser- 
vice providers have other motives besides profit-seeking for 
being in the fields they’re in. If a hospital is losing patients 
because it provides poor care, that loss of patients will pro- 
vide a motive to improve care regardless of whether the hos- 
pital is for-profit or non-profit— and the patients know it. So 
it isn’t as though people are only accustomed to thinking this 
way about profit-seeking businesses. 

This lack of connection between what Americans think about 
choice and competition in K-12 education and what it thinks 
about choice and competition in virtually every other aspect 
of life is a great hindrance to accurate public discussion of 
school vouchers. One good hope for rectifying that problem 
is to make the public aware of the large body of empirical re- 
search that examines how vouchers impact public schools. 



Why Scientific Methods Matter 

When evaluating the effectiveness of an education policy, it 
is especially important to rely on empirical research of high 
scientific quality. Otherwise, it is very difficult to determine 
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the effects of an education policy. Student outcomes are 
affected by so many different influences— including demo- 
graphic factors (income, race, family structure, etc.), school 
factors (type of school, teacher quality, etc.) and intangibles 
such as the level of enthusiasm parents and teachers invest 
in a child’s education. The job of empirical science is to dis- 
entangle the influence exercised by each of these factors as 
well as can be done with the available evidence. 

A study that uses good methods can overcome these problems 
and provide reliable information about what is influencing 
student outcomes. But if scientific procedures are not rigor- 
ously followed, or if we don’t bother looking at the science and 
try to make judgments without it, we can come to the wrong 
conclusions about what factors cause what outcomes. 

The gold standard for empirical science is the method known 
as “random assignment,” in which subjects are randomly di- 
vided into a treatment group that will receive the treatment 
being studied (such as vouchers) and a control group that 
will not receive it. Because the two groups are separated 
only by a random lottery, they are likely to be very similar 
in every respect other than the treatment. 

It is usually not possible to conduct random-assignment re- 
search on education policy. School vouchers have been one of 
the rare exceptions, because when there are more applicants 
for a voucher program than there are slots available, a ran- 
dom lottery is often used to determine who may participate, 
creating a naturally occurring random-assignment research 
design. There is a substantial body of random-assignment 
research on the academic achievement of students who are 
offered vouchers, and it consistently finds that vouchers im- 
prove student achievement.^ 

But while it may be the best kind of research, random-assign- 
ment research is not the only kind of research worth consider- 
ing. Where it is not possible to conduct a random-assignment 
study, other kinds of research methods can produce useful 
information that sheds light on important policy questions. 

The next best research method is to track year-to-year chang- 
es in outcomes for individual students. Tracking individual 
students over time removes from the analysis most, though 



not all, of the influence of unmeasured factors. If a student 
is advantaged in a way that is not measurable, that advan- 
tage will typically be present in the student’s outcomes for 
both year one and year two of the study; thus the change in 
outcomes between year one and year two will mostly be due 
to other factors— though unmeasured factors will still exert 
some influence on the level of year-to-year change. Remov- 
ing the influence of unmeasured factors allows the analysis 
to isolate the impact of the factors that are being measured, 
such as exposure to vouchers. 

If it is not possible to track individual students, good research 
still can be done by tracking year-to-year changes in indi- 
vidual schools. It is reasonable to expect that the unmeasured 
advantages of the students in a given school will be similar 
from year to year. If a school has highly advantaged students 
in 2006, it probably will still have highly advantaged students 
in 2007. Mobility among the student population will create 
some change in student characteristics from year to year, but 
not so much that we cannot learn from school-level studies. 
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THE EVIDENCE ON VOUCHER PROGRAMS 




Choice and competition 
provided by vouchers 
improve public schools. 



A total of 17 empirical studies have been conducted on how 
voucher programs impact academic achievement in public 
schools. Of these studies, 16 find that vouchers improve pub- 
lic schools. The one remaining study found that vouchers 
had no visible impact on public schools. Significantly, that 
one study was also the only study conducted on a voucher 
program that intentionally protects public schools from the 
impact of competition. Thus, it does not detract from the 
very strong research consensus that choice and competition 
provided by vouchers improve public schools. 

Milwaukee Vouchers 

A total of five empirical studies have been conducted on how 
the Milwaukee voucher program affects academic outcomes 
at public schools. All five unanimously find that vouchers 
improve Milwaukee public schools. 

Vouchers are available to all Milwaukee students who meet 
certain criteria, most notably an income restriction. Thus, in 
Milwaukee there is not a simple division between public schools 
that are and are not exposed to vouchers, as in some other 
programs. However, some Milwaukee public schools are much 
more exposed to vouchers than others, based on the demo- 
graphic makeup of their student populations. Thus, research- 
ers have focused on isolating the academic impact of a school’s 
being more exposed to vouchers versus being less exposed. 

The first empirical study on the Milwaukee program was con- 
ducted by Caroline Hoxby, then of Harvard University, and re- 
leased in 2001. She compared schools where at least 66 percent 
of the student population was eligible for vouchers to schools 
where fewer students were eligible for vouchers. She found that 
in a single year, schools in the “more exposed to vouchers” 
group made gains that were greater than those of other Mil- 
waukee public schools by 3 percentile points in math, 3 points 
in language, 5 points in science and 3 points in social studies.^ 
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The next study, released in 2002, was conducted by Jay Greene 
and Greg Forster, then of the Manhattan Institute. Rather 
than dividing Milwaukee public schools into two groups, they 
used regression analysis to determine how changes in the 
percentage of students in a Milwaukee public school who 
were eligible for vouchers would impact a school’s academic 
results. They found that greater exposure to vouchers had 
a positive effect on year-to-year changes in public school 
outcomes; the size of the effect was such that a school with 
all its students eligible for vouchers could be expected to 
outperform a school with only half its students eligible by 15 
percentile points over four years.® 

In two analyses that were released in 2006, Rajashri Chakrabar- 
ti of the Federal Reserve Bank found that the Milwaukee 
voucher program improved public schools. Chakrabarti con- 
ducted multiple analyses using different methods for mea- 
suring public schools’ exposure to vouchers. Some are simi- 
lar to Hoxby’s method (though Chakrabarti divided schools 
into three groups rather than two) and others to Greene and 
Forster’s method. In both studies, Chakrabarti found that 
increased exposure to vouchers improves academic gains in 
Milwaukee public schools. A revised version of one of these 
studies was released in 2008.^ 

Finally, a 2007 study was conducted by a team of research- 
ers led by Martin Carnoy of Stanford University. This study 
used a modified form of the Hoxby/Chakrabarti method. The 
authors reported that their analysis “confirms the earlier re- 
sults showing a large improvement in Milwaukee in the two 
years following the 1998 expansion of the voucher plan to re- 
ligious schools.” Before 1998, religious schools were excluded 
from the Milwaukee program, so many fewer students par- 
ticipated. When religious schools were admitted to the pro- 
gram in 1998, participation increased dramatically.® 



Curiously, the authors of this study seemed anxious to deny 
the positive results of their own study. Although their data 
clearly showed that the Milwaukee voucher program im- 
proved public schools, the authors argued that we should not 
conclude that the voucher program improved public schools. 
They based this argument on two additional results. First, 
they found that in later years, after the period immediately 
following the program’s expansion, the positive effects of the 
program did not get bigger (although the original positive 
effect observed in the 1998-2000 period was sustained in those 
subsequent years). Second, they found that academic results 
in Milwaukee public schools were not visibly related to the 
concentration of private schools nearby. 

Neither of these is a legitimate reason to deny the study’s 
analysis showing that the Milwaukee voucher program im- 
proved public schools. If the improvements did not get big- 
ger after 1998-2000, that does not mean the program failed to 
improve public schools. It just means that the improvements 
were not cumulative.^ Given that the Milwaukee program re- 
mains cramped by numerous restrictions on its operation 
even after the 1998 expansion, there is plenty of room for 
further expansion of the program. If the 1998 expansion pro- 
duced a positive response, the findings of the Carnoy et. al. 
study (and those of all the other research in Milwaukee) 
suggest that a further expansion of the program would be 
a promising way to produce even bigger improvements in 
Milwaukee public schools. 

And if the presence of private schools nearby was not a 
factor in improving public schools in Milwaukee, this only 
means that the positive effects of the voucher program are 
not dependent upon geography— neighborhoods that did not 
have high concentrations of private schools still benefited 
from the voucher program. 
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Florida Vouchers 

A total of ten empirical studies have been conducted on how 
Florida’s two voucher programs have affected academic 
outcomes at public schools. All ten unanimously find that 
vouchers have improved Florida public schools. 

Nine of these Florida studies examine the effects of the 
state’s A+ program, which gave vouchers to students at 
chronically failing public schools, before the program was 
ended by court order in 2006. In addition, one empirical 
study— the most recent one— examines how Florida’s Mc- 
Kay voucher program, which provides vouchers to disabled 
students in public schools, affects academic outcomes for 
disabled students who do not use the program and remain in 
Florida public schools. 

Under the A+ program, each public school received an annual 
grade from the state based primarily on how many of its stu- 
dents either achieved an adequate score on the state test or 
made substantial progress toward an adequate score. If a school 
received two (or more) F grades from the state in any four-year 
period, students who had attended that school in the year of its 
second (or subsequent) F grade could apply for vouchers. Stu- 
dents were required to apply for vouchers during the two-week 
period immediately following the public announcement of the 
second (or subsequent) F grade; after this brief window closed, 
vouchers were no longer available. However, those students who 
did manage to apply during the brief eligibility window could 
continue using vouchers in subsequent years. 

The first study of the A+ program was published in 2001 by 
Greene. At that point, only two schools had ever been eligible 
for vouchers under the program— too few to provide a basis 
for meaningful analysis. Instead, Greene studied the impact 
of the mere threat of vouchers on schools that were in danger 
of becoming eligible for vouchers if they did not improve. 

Using a simple descriptive analysis, Greene found that 
schools that had received an F grade, which would be eligible 
for vouchers if they received another F grade, made much 
larger year-to-year gains than schools that received a D (18 
points in reading and 26 points in math for F schools versus 
10 points in reading and 16 points in math for D schools). 
Greene then drew two further comparisons intended to iso- 



late the impact of the voucher threat: high-scoring F schools 
compared with low-scoring D schools, and high-scoring F 
schools compared with low-scoring F schools. There was a 
substantial difference between high-scoring F schools and 
low-scoring D schools (16 points in reading and 24 in math 
versus 13 points in reading and 18 in math). However, a re- 
gression analysis showed that among F schools there was no 
statistical relationship between their test scores in the prior 
year and their test scores in the subsequent year— high- 
scoring F schools and low-scoring F schools had about the 
same results in the following year. Greene concluded that 
the difference in outcomes was attributable to receiving an 
F grade from the state, which included the voucher threat.^® 

This analysis was methodologically simple, as is often the 
case the first time an empirical question is being studied. 
Greene’s analysis in this first study did not examine some al- 
ternative possibilities that might account for a relationship 
between receiving an F grade and making bigger improve- 
ments the next year. His next study, and later studies con- 
ducted by others, included additional analyses designed to 
test whether the improvements associated with the F grade 
were due to these alternative explanations or to vouchers, or 
both. (See the “Alternative Theories” section below for dis- 
cussion of these alternatives and the results of the analyses 
examining them.) 

In a subsequent study, along with Marcus Winters of the 
Manhattan Institute, Greene used a more advanced statisti- 
cal method. Greene and Winters divided schools into four cat- 
egories. Sometimes D schools were those that had received 
a D grade, but no F grades and at least one grade above a 
D, in any of the previous four years; Always D schools were 
those that had received D grades in each of the previous 
four years; Voucher Threatened schools were those that had 
received exactly one F grade in the previous four years; and 
Voucher Eligible schools were those that had received two or 
more F grades in the previous four years. They then used re- 
gression analysis to compare the year-to-year gains made in 
schools in each of these four categories with those of other 
Florida schools. 

For both math and reading scores, on both the state test and 
the national norm-referenced Stanford-9 test, Greene and 
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Winters found that the positive impact of the A+ program 
closely tracked the schools’ distance from vouchers. Voucher 
Eligible schools made the biggest academic gains, followed by 
smaller gains in Voucher Threatened schools, followed by the 
two categories of schools that had received Ds but no Fs. For 
example, in math scores on the state test. Voucher Eligible 
schools made improvements 15 points higher than other Flor- 
ida public schools, while Voucher Threatened schools made 
improvements 9 points higher. Always D schools 4 points high- 
er, and Sometimes D schools 2 points higher. 

When Greene and Winters’ analysis was published in the 
journal Education Next in the summer of 2004, it was accom- 
panied by an analysis conducted by Rajashri Chakrabarti, 
then of Cornell University. Chakrabarti used a simple de- 
scriptive analysis to provide further assurance that the rela- 
tionship between the F grade and school improvements was 
due to vouchers. She compared the improvements made by 
F schools under the A+ program with improvements made 
by schools in the lowest performance category (out of four) 
under the state’s previous school evaluation system. The 
previous system had no voucher component. Chakrabarti 
found that under the previous system, putting a school in 
the lowest-performing category did not improve its perfor- 
mance relative to schools in the next lowest performance 
category, while F schools did make bigger gains than those 
of D schools under the A+ program. Over three years, the 
gap between F schools and D schools closed from almost 15 
points to about 5 points.^^ 

In 2006, Chakrabarti released a more sophisticated analy- 
sis that compared the impact of the A+ program on public 
schools to that of the Milwaukee voucher program (this is 
one of the two Milwaukee studies by Chakrabarti cited in the 
section on the Milwaukee program, above). A revised ver- 
sion of the study was released in 2008. 

In this study, she used regression analysis to compare the 
trends over time in the academic achievement of schools 
that received F, D, and C grades in 1999, the first year of 
the A+ program. She compared trends in outcomes at these 
schools before and after the implementation of the program 
in 1999. Chakrabarti found that when F and D schools are 
compared to each other, the F schools made gains 8 points 



larger in reading and 5 points larger in math over three 
years. When F and D schools are separately compared to C 
schools, the F schools made gains 17 points larger in reading 
and 11 points larger in math than the C schools over three 
years, while D schools made gains 9 points larger in reading 
and 4 points larger in math than the C schools. 

Chakrabarti then confirmed the impact of the F grade using 
a method known as “regression discontinuity,” which iso- 
lates the impact of the F grade from other factors by com- 
paring high-scoring F schools with low-scoring D schools. 
Regression discontinuity is a very high-quality method that 
is widely considered the next-best thing to a random-assign- 
ment study. However, it does limit the scope of the analysis, 
since it excludes many of the F schools from the data set. 
She found that the high-scoring F schools outscored the low- 
scoring D schools by 7 points in reading and 6 points in math 
over three years. 

Chakrabarti further confirmed this analysis with a follow- 
up study in 2007. This study took advantage of the fact that 
school grades are based primarily on how many students are 
either above or approaching a given cutoff score on the state 
test. Chakrabarti found that in schools that had received an 
F grade, students near the cutoff made larger gains relative 
to the gains of students at other schools, while other students 
at F schools were not negatively affected. The study used a 
regression discontinuity design to compare high-scoring F 
schools and low-scoring D schools.^^ 

David Figlio of the University of Florida and Cecelia Rouse 
of Princeton University have also studied the A+ program to 
examine its impact on public schools. Their initial analysis, 
released in 2004, was the first to use student-level data rather 
than school-level data, providing improved scientific quality. 
In this analysis they examined data up through 2000-01 (that is, 
before vouchers became widely available in 2002-03, as was the 
case in Greene’s initial 2001 study). They found that if a school 
received an F grade, its students made gains on the state test 
that were 2 points larger in reading and 5 points larger in math 
than those of other Florida schools over one year. Scores on the 
national Stanford-9 test also improved. They confirmed the ex- 
istence of a positive effect from the F grade using a regression 
discontinuity model, examining scores on the Stanford-9.^^ 
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In a subsequent study released in 2007, in which they were 
joined by Jane Hannaway of the Urban Institute and Dan 
Goldhaber of the University of Washington, they collected 
data up through 2004-05, using these data to track the con- 
tinuing effects on schools that had received Fs in 2002-03. 
The study used a regression discontinuity model to compare 
high-scoring F schools and low-scoring D schools. It found 
that receiving an F grade in 2002-03 produced academic im- 
provements in students’ test scores in the next year relative 
to those in non-F schools, and that these improvements were 
sustained in future years. They presented their results in 
terms of standard deviations rather than test score points; 
they found that the gains were equal to about a tenth of a 
standard deviation.^® 

Martin West and Paul Peterson of Harvard University re- 
leased an analysis that also used individual student data in 
2005. It found that among schools that had not received the 
lowest possible rating under the state’s previous school eval- 
uation system (and thus were “shocked” by the imposition of 
the F grade), receiving an F under the new accountability 
system produced an improvement in student’s test scores 
equal to about four percent of a standard deviation.!^ 

In 2008, Forster, now of the Friedman Foundation, conducted 
a study examining the impact of the A+ program in every 
year from 2001 through 2006. Previous studies had only ex- 
amined the impact of getting a particular grade, such as 
F or D, in a single year (usually either 1999, the first year 
grades were given out under the A+ program, or 2002, the 
first year when a substantial number of schools were eligible 
for vouchers). Because vouchers were not widely available 
until 2002, and the voucher element of the A+ program was 
struck down by court order in early 2006, this study was able 
to track the changing impact of the A+ program as the sta- 
tus of the vouchers in the A+ program changed. 

Forster used Greene and Winters’ four categories to exam- 
ine the impact of the voucher threat. He found that in 2001, 
before vouchers were widely available. Voucher Threatened 
schools made gains relative to all Florida schools equal to 13 
points on Florida’s new “developmental scale,” which uses a 
single scale to track student scores from 3rd grade through 
high school. The next year, when vouchers were widely avail- 



able, Voucher Threatened schools gained 15 developmental 
points, but Voucher Eligible schools gained 67 developmental 
points relative to other Florida schools. Over the next three 
years, as the percentage of eligible families using vouchers 
decreased due to the artificial obstacles created by the state 
department of education, the positive voucher effect was not 
as large but remained substantial (Voucher Eligible schools 
gained from 20 to 27 developmental points each year). Then, 
in 2006, the first year after the voucher element of the pro- 
gram was removed. Voucher Eligible schools gained only 11 
developmental points. Results for Voucher Threatened and D 
schools followed similar patterns.^^ 

The only empirical study conducted on Florida’s McKay 
voucher program was also released in 2008. The McKay pro- 
gram allows any disabled student in Florida public schools to 
use a voucher. Conducted by Greene and Winters, now of the 
University of Arkansas, the study used student-level data over 
five years to measure the relationship between the academic 
performance of disabled students in public schools and the 
number of private schools accepting McKay vouchers nearby. 
While the Carnoy, et. al. study in Milwaukee found no rela- 
tionship between the presence of private schools nearby and 
voucher improvements (though it did find that exposure to 
vouchers through student eligibility produced improvements), 
the Greene and Winters study found a strong relationship be- 
tween the presence of private schools participating in the Mc- 
Kay program and voucher improvements. 

The strongest effect of the McKay program was among students 
classified as learning disabled, representing 61 percent of all Flor- 
ida disabled students. At a public school with an average number 
of private-school McKay competition within five miles, the posi- 
tive impact of the McKay program was equal to 16 points in math 
and 24 points in reading among learning disabled students.^^ 

Other Programs 

Four studies have been conducted on the impact of voucher 
programs in other places. Three of these studies find that 
vouchers improve public schools; one finds that vouchers 
make no visible difference to public school outcomes. 

The first of these studies, in 2002, was conducted by Christopher 
Hammons of Houston Baptist University. Hammons examined 
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century-old voucher programs in Maine and Vermont. When 
these states first created public schools, they gave small towns 
the option of “tuitioning” their students - using public funds to 
pay for their students to attend private schools or nearby pub- 
lic schools - rather than building their own public schools. 

Hammons measured the relationship between a public 
school’s academic achievement and its distance from the 
nearest “tuitioning” town. Using regression analysis, he 
found a positive relationship. The relationship was strong 
enough that if a town one mile away from a school began tu- 
itioning its students, the percentage of students at the school 
passing the state’s achievement test could be expected to go 
up by 3 percentage points.^^ 

In the same 2002 study in which they examined the impact 
of the Milwaukee program (see above), Greene and Forster 
also examined the impact of a large-scale privately funded 
voucher program targeted to the Edgewood school district, 
in San Antonio, Texas. Unfortunately, it was not possible to 
differentiate between Edgewood schools that were more or 
less exposed to competition from the voucher, because the 
program offered vouchers to every student in the Edgewood 
district. Greene and Forster instead examined the perfor- 
mance of the district as a whole. 

Controlling for demographics and local resources, they found 
that Edgewood’s year-to-year test score gain outperformed 
those of 85 percent of school districts in Texas. Given that Edge- 
wood is a high-poverty (93 percent eligible for lunch programs) 
and high-minority (97 percent Hispanic) district, the study con- 
cludes that such a high statewide academic rank for Edgewood 
suggests that vouchers produced public school improvements.^^ 

In 2006, Greene and Winters released a study of how the fed- 
eral voucher program in Washington, D.C. impacts public 
schools. Because eligibility for the voucher program is re- 
stricted to a relatively small number of students, particularly 
in the program’s first year (when the study was conducted), 
Greene and Winters measured exposure to the voucher pro- 
gram by measuring the distance between each public school 
and the nearest private school participating in the voucher 
program. They found no visible relationship.^^ 



The D.C. voucher program is the nation’s only voucher pro- 
gram with a “hold harmless” provision that allocates addi- 
tional money to the public school system to “compensate” for 
the loss of students. This is intended to insulate the public 
school system from the impact of competition from vouch- 
ers. Thus, the absence of a visible effect in this study does 
not detract from the research consensus in favor of a posi- 
tive impact from vouchers on public schools. 

Finally, in a 2008 study, Forster examined the impact of 
Ohio’s EdChoice voucher program on public schools. The 
EdChoice program offers vouchers to all students attending 
chronically failing public schools. In the program’s first year 
—the year covered by Forster’s study - schools were eligible 
if they had been designated in a state of “academic emergen- 
cy” by the state in each of the last three years. The definition 
was subsequently expanded to include more schools. 

Forster used regression analysis to examine year-to-year test 
score changes in schools where students were eligible for vouch- 
ers. Forster found positive effects from the EdChoice program 
in math scores for 4th and 6th grade students and reading scores 
for 6th grade students, and no visible effect in other grades. The 
positive effects ranged from 3 to 5 points in one year.^^ 
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ALTERNATIVE THEORIES 




The body of high-quality research 
that consistently shows that 
vouchers improve public schools has 
grown ever larger over the years. 



The body of high-quality research that consistently shows that 
vouchers improve public schools has grown ever larger over the 
years. It has also grown more methodologically sophisticated. As 
the first studies on this subject emerged, some speculated that the 
improvements in public schools might be caused by other factors 
besides positive incentives from vouchers. The empirical research 
did not ignore these questions, but rigorously tested the alternative 
hypotheses that were offered. 

Is It Vouchers or Student "Dredging''? 

One alternative explanation for the positive results in these stud- 
ies is the change of composition in the student population caused 
by vouchers. This theory speculates that the worst students may 
be using the vouchers, leaving behind the better students in pub- 
lic schools. This would increase the level of academic achieve- 
ment among public school students without actually improving the 
schools. Moreover, some believe that students exert strong “peer ef- 
fects” on one another’s performance; on this theory, the removal of 
the worst students from public schools might produce a multiplier 
effect, causing student achievement to go up significantly. 

For this theory to be true, vouchers would have to be attracting 
participants disproportionately from among the lowest performing 
students. Instead of taking away the best students, as so many op- 
ponents of vouchers claim, on this theory vouchers would be taking 
away the worst students. So while some critics claim that vouchers 
“cream,” others claim that they “dredge.” 

The direct evidence on this question supports neither the creaming 
nor the dredging hypothesis. As has already been noted above, stud- 
ies directly comparing voucher applicants with the population of 
students who were eligible to apply have found that the applicants 
were very similar to the eligible population not only in terms of de- 
mographics, but also in terms of educational achievement. If vouch- 
ers were either creaming or dredging, this would not be the case.^^ 
While it would be good to have more evidence on this question, the evi- 
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dence that we do have does not support either creaming or dredging. 

However, it is also worth looking more broadly at the evidence pro- 
vided by studies on the impact of voucher programs. Comparing 
the characteristics of voucher applicants to those of the eligible 
population is difficult, because it is cumbersome to collect data 
on the eligible population that does not apply. There are therefore 
fewer high-quality studies that directly measure this comparison, 
so we should also consider what the broader body of evidence indi- 
cates about this question. 

Perhaps the most important piece of additional evidence in the broad- 
er literature is the impact Florida’s A + program has on public schools 
that are merely threatened with vouchers. There has been no move- 
ment of students in these schools, yet vouchers have a positive impact. 

It is also worth noting that a number of studies in Florida have 
tracked the achievement of individual students. These studies 
would not be misled by the immediate impact of population shifts, 
since they follow students rather than whole schools. It is true that 
these studies would not exclude any peer effects, if such effects 
exist. But in order to rescue the “dredging” hypothesis, we would 
have to conclude that the entire positive effect from vouchers in all 
these studies was attributable solely to peer effects. Given that the 
direct evidence finds no such “dredging” occurring in cases where 
we are able to measure it, and that the studies on Florida’s A+ pro- 
gram find a positive voucher impact even where no students have 
changed schools, this is not a remotely plausible hypothesis. 

Related to the issue of changes in student composition is the issue 
of changes in school resource allocation. As has been noted above, 
school choice doesn’t drain money from public schools, it improves 
their financial situation by leaving behind more resources to serve 
fewer students. If we wanted to deny the existence of a positive 
effect from voucher competition, we might attribute the positive 
effects of vouchers to this fiscal benefit. 



It is worth bearing in mind that this would not be an argument 
against vouchers even if it were true, because if vouchers only 
benefited public schools by saving them money, they would still be 
benefiting public schools. However, as we have seen, the evidence 
in Florida shows a positive voucher effect even where no students 
have actually changed schools, simply as a result of the threat of 
vouchers. Thus, the evidence consistently shows that vouchers im- 
prove schools through competition as well as by saving them money. 

Is It Vouchers or a Stigma Effect? 

After Greene’s initial study of Florida’s A+ voucher program, two 
alternative theories were offered that would attribute the study’s 
positive finding to factors other than the influence of vouchers. 
One of these is the hypothesis that Greene’s study was observing a 
“stigma effect.” The stigma hypothesis is that schools assigned an 
F grade by the state improve in order to remove the stigma of being 
labeled as failing, rather than responding to voucher competition. 

Before considering how subsequent studies have examined this hy- 
pothesis, it is worth noting that stigma is not a possible explana- 
tion for the positive findings of all five studies conducted on vouch- 
ers in Milwaukee, the study of Florida’s McKay program, or the 
study of town tuitioning vouchers in Maine and Vermont. In these 
programs, public schools were exposed to vouchers without being 
publicly stigmatized. Thus, there is no way to attribute the positive 
impact associated with these voucher programs to stigma. 

In their follow-up study, Greene and Winters checked for a stig- 
ma effect in Florida by examining schools that had received an F 
five years previously, but no Fs in the preceding four years. These 
schools were threatened with vouchers if they got another F in the 
three years following their first F. However, in the final year ob- 
served by the study, these schools were no longer threatened with 
vouchers if they got an F. However, these schools were still schools 
that had been labeled as failing. Greene and Winters found that, 
rather than seeing any gains associated with the stigma effect. 
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these schools actually backslid a little, relinquishing some of the 
gains associated with being a Voucher Threatened school. 

Other studies have tested the stigma hypothesis in other ways. 
Chakrabarti’s 2004 analysis, published alongside Greene and 
Winters’ study in Education Next, compared the performance of 
schools placed in the lowest performance category under the A+ 
program and under the state’s previous school evaluation program. 
Since the previous program included no vouchers, it provided an 
opportunity to test for the presence of a stigma effect without the 
possibility of confusion between a voucher effect and a stigma ef- 
fect. As has been outlined above, her comparison showed that the 
previous program, without vouchers, did not close the gap between 
the lowest and second-lowest performance categories, indicating 
that there was no positive effect from being placed in the lowest 
category. Conversely, under the A+ program, schools in the lowest 
category did make substantial progress in closing the gap with the 
next-lowest category. 

In her 2006 study comparing Florida’s A+ vouchers to the Milwau- 
kee voucher program, Chakrabarti used regression analysis to test 
the validity of this observation. She found that under the previous 
school evaluation program, schools in the lowest-performing cat- 
egory did not close the gap with either the second-lowest or third- 
lowest performance groups, indicating that there was no positive 
effect from being placed in the lowest category. She expanded this 
analysis in her 2007 study to examine not only whether there was 
an impact on the average score, but also on whether there was an 
impact on how many students fell into each performance category 
on the exam. Her findings were again negative. 

In the 2007 study she also conducted a different test, examining 
the relationship between the concentration of private schools near 
an F school just before the voucher program was initiated and the 
strength of the positive effect of the program on F schools. She 
found that F schools that had more private schools nearby when 
the program began responded to the program more strongly, im- 
plying that the positive results of the program are due to competi- 
tion and not a stigma effect. 

The 2004 Figlio and Rouse study also tested for stigma by compar- 
ing the performance of F schools under the A+ plan to the perfor- 
mance of schools in the lowest performance category under the 
state’s previous school evaluation program. Unlike Chakrabarti’s 
two later studies, this study did find that putting schools into the 



lowest performance category under the previous program had a 
positive impact on outcomes. However, the impact of the previous 
program was not as large as the impact of the A+ program. Figlio 
and Rouse concluded that the A+ program has a positive impact on 
schools both because of a stigma effect and because of competition 
from vouchers. They estimated that the stigma effect was larger than 
the competitive effect, but their findings affirmed that both exist. 

Finally, Forster’s 2008 study of the A+ program tested the stigma 
hypothesis in a very different way. Forster separately examined the 
impact of receiving an F grade in each year from 2001 through 2006. 
He found that the impact varied in size dramatically depending on 
the year— from a very large impact in the first year vouchers were 
widely available to a more moderate impact in later years, as the 
percentage of eligible families using the voucher went down due 
to obstacles created by the state department of education, and an 
even more moderate impact after vouchers were removed from the 
program. Since the stigma of getting an F grade did not change 
from year to year, but the status of vouchers in the A -k program 
did, the dramatic changes in the impact of the program cannot 
plausibly be attributed entirely to stigma. 

Given that other voucher programs show a positive effect without 
the possible presence of stigma, and that all studies of the A+ pro- 
gram that have specifically examined the question have concluded 
either that there is no stigma effect or that the stigma effect co- 
exists with a competitive voucher effect, there do not seem to be 
reasonable grounds for attributing the positive results from the A+ 
program to a stigma effect. 

Is It Vouchers or Regression to the Mean? 

The other alternative theory that emerged after Greene’s origi- 
nal study of the A+ program is the “regression to the mean” or 
“mean reversion” hypothesis. Regression to the mean, also known 
as mean reversion, is a statistical phenomenon under which fail- 
ing schools are more likely to improve than to get worse simply 
because they can’t much worse than they already are. When a vari- 
able approaches zero it becomes more likely to go up rather than 
down, simply because there’s less room to go down before it hits 
the “floor” of zero. 

As with the stigma hypothesis, we should note that other voucher 
programs have shown positive effects without a strong possibil- 
ity of confusion caused by regression to the mean. The improve- 
ments in public schools caused by Milwaukee’s voucher program. 
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unanimously affirmed by all five studies in that city, are unlikely to 
be caused by regression to the mean. While the schools that were 
more exposed to vouchers did start out lower on the scale of aca- 
demic achievement than those less exposed to vouchers, they were 
not the “lowest of the low,” schools singled out because of chronic 
failure, as in the A + program. The results from Florida’s McKay 
program and from town tuitioning vouchers in Maine and Vermont 
are even more clearly not caused by regression to the mean; in 
these cases the exposure to vouchers is not systematically related 
to the academic performance of the schools. 

In their follow-up study, Greene and Winters tested for regression 
to the mean in the A+ program by comparing F schools to oth- 
er schools that had very similar test scores but had not received 
Fs. These schools did not receive Fs because the school grades 
were not exclusively based on test scores. They found that the F 
schools made greater gains even though both types of schools (the 
F schools and the similarly-scoring schools that did not receive Fs) 
were equally subject to the possibility of regression to the mean. 

In their 2004 study, Figlio and Rouse tested for regression to the 
mean by examining the behavior of test scores at low-scoring 
schools in 1995, before either the A+ plan or the previous school 
evaluation system was in place. They found that schools that would 
have been labeled as failing schools under either of these later sys- 
tems did not exhibit any visible tendency towards rising test scores. 
Thus, the rising test scores observed under the later systems can- 
not be attributed to regression to the mean. 

West and Peterson, in their 2005 study, tested for regression to the 
mean by conducting two additional comparisons that measured the 
performance of F schools as compared to two groups of low-scoring 
D schools (selected by two different criteria for what counted as 
“low-scoring”). In both these comparisons they found that the ac- 
tual baseline test scores - their academic achievement at the time 
when the grades came in - of the F schools and the comparison D 
schools were not statistically different (that is, there were no “sta- 
tistically significant” differences). Yet in both comparisons the F 
schools outperformed the D schools, signaling that getting an F had 
a positive impact that was not attributable to regression to the mean. 

Chakrabarti’s 2006 and 2007 studies tested for regression to the 
mean in the A + program using two methods. First, she used 1998 
data and assigned schools the letter grades they would have got- 
ten if the A + program had been in effect in that year, and tested 



to see whether “1998 F” schools made gains relative to “1998 D” 
schools and “1998 C” schools. They did not. Her second method was 
to compare results in F schools and D schools in a single subject 
(reading, math or writing) where those schools scored similarly. 
Since a school’s grade is based on its performance across multiple 
subjects, it is possible to identify a set of F schools and D schools 
that are similar in their performance in one given subject. This 
analysis also found that regression to the mean was not driving the 
study result. The 2006 study also tested its Milwaukee analysis for 
regression to the mean by examining the behavior of test scores 
before the voucher program was implemented. Here the analysis 
also found no evidence of regression to the mean. 

Forster’s 2007 study of the A+ program excludes regression to the 
mean in the same way it excludes the stigma hypothesis: by com- 
paring results in each year from 2001 to 2006. Over this period, the 
impact of the voucher program varied considerably, and the varia- 
tions tracked the changing status of vouchers in the program. Re- 
gression to the mean, by contrast, would have been roughly equally 
present in all years. The existence of variations, and especially of 
variations that track the changing status of vouchers, renders it im- 
plausible that regression to the mean could be driving the results. 

In addition, Forster’s 2008 study of Ohio’s EdChoice program, the 
only study of that program to date, checked for regression to the 
mean. Forster conducted a second analysis in which his data set 
included only schools located in school districts that had been la- 
beled “major urban - very high poverty” by the state. The results of 
the regression analysis were virtually identical as the results from 
the original, statewide analysis. That there was no change in result 
between the analysis that included all schools and the analysis that 
included only schools located in the most troubled districts suggests 
that regression to the mean was not driving the results. 

Perhaps most important, the studies using regression discontinu- 
ity (Figlio and Rouse 2004, Chakrabarti 2006, Chakrabarti 2007 and 
Rouse, Hannaway, Goldhaber and Figlio 2007) all confirmed the 
positive effect from the voucher program. A regression disconti- 
nuity design excludes regression to the mean because the schools 
in the treatment group (high-scoring F schools) and the control 
group (low-scoring D schools) begin with similar test scores. The 
influence of regression to the mean will be similar on both groups, 
so any systematic differences in results can be attributed to the 
vouchers and not to regression to the mean. 
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WHYAREN'TPUBLICSCHOOLS FIXED YET? 




The empirical evidence 
consistently shows that vouchers 
have succeeded in improving 
public schools. 



Among those who wish to distract the public from this large 
body of high-quality scientific evidence, one of the most com- 
mon strategies is to complain that public schools in places 
like Milwaukee are still failing to educate so many of their 
students. Milwaukee public schools were widely dysfunc- 
tional in 1990 when the voucher program was enacted, and 
they remain widely dysfunctional today. There has been no 
“Milwaukee miracle.” 

But the absence of a dramatic “miracle” is not a valid reason 
to conclude that vouchers aren’t helping. Just because a car 
can’t go 700 miles per hour doesn’t mean it can’t go 70 miles 
per hour, and a man who cannot walk on water may still 
be able to swim. The empirical evidence consistently shows 
that vouchers have succeeded in improving public schools. 
It is not difficult to see how vouchers might have a positive 
effect without working miracles. 

Other Factors Besides Vouchers Affect Schools 

The overall performance of a school system is affected by 
countless factors. Some of these factors, such as political 
policymaking, can change quickly and dramatically. As a 
result, the overall performance of a school system can never 
by itself provide a reliable guide to whether any one factor 
(such as vouchers) is having a positive effect. If a man with 
asthma starts taking a new medication, and at the same time 
takes up smoking, his overall health and ability to breathe 
may not improve but this has no bearing on the question of 
whether the medicine is helping. 

The only way to know whether vouchers are having a posi- 
tive impact is to conduct empirical research using high-qual- 
ity scientific methods. The whole purpose of these studies is 
to isolate the impact of vouchers from the impacts of all the 
other factors that influence academic outcomes, so that we 
can measure it accurately. Given the remarkably unanimous 
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research supporting the positive impact of vouchers every- 
where they are allowed to affect public schools, to respond 
by holding up the continued failure of public schools overall 
is simply obscurantism. 

Voucher Programs Are Heavily Restricted 

The positive impact of voucher programs on public schools 
identified in the empirical research is sometimes modest in 
size. That is hardly surprising, given that existing voucher 
programs are also modest in size. If modest programs pro- 
duce modest benefits, not dramatic benefits, is the logical 
conclusion to deny that voucher programs have any benefits 
and give up on them? Or to expand them until they are large 
enough to have a dramatic impact? 

Existing voucher programs are hindered by limits on the 
number of students they may serve, limits on the types of 
students they may serve, limits on the purchasing power they 
are allowed to provide, limits on families’ ability to supple- 
ment that purchasing power, limits on how students may be 
admitted to participating schools, and so forth. These limits 
are detailed in the 2008 Friedman Foundation report “Grad- 
ing School Choice.” 

Yet some people demand that these programs, while still re- 
stricted in size and scope, must produce Herculean results or 
else be given up as hopeless. No matter how clear the scien- 
tific evidence showing that vouchers improve public schools, 
we still hear the response that public schools haven’t yet 
been radically transformed in places with vouchers, there- 
fore vouchers had no effect. 

Some of the most restrictive limits are imposed in the Mil- 
waukee voucher program. That program was the early 
pioneer that founded the modern school choice movement. 
Because it started at a time when vouchers did not have 



a national movement behind them, the Milwaukee program 
had to accept more political compromises than recent school 
choice programs have had to accept. For the same reason 
—because it was the original pioneer— the Milwaukee pro- 
gram is taken to be the flagship voucher program and is the 
nation’s most prominent school choice program. So, ironi- 
cally, the program that labors under the worst restraints, 
which therefore ought to be expected to produce especially 
modest results, is actually expected to produce a “Milwau- 
kee miracle” and vindicate all vouchers everywhere. This is 
simply unreasonable. 
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CONCLUSION 




Vouchers do, in fact, improve 
public schools. 



Even if vouchers did not improve public schools, there would 
still be other reasons to implement them. They provide a 
better education to those who use them, they provide better 
services for disabled students, they put students into schools 
that are more racially integrated, they improve students’ 
civic values, they save the public money, and so forth.^^ 

But vouchers do, in fact, improve public schools. Of the 
seventeen empirical studies conducted on this question, 16 
find that vouchers improve public schools and one— the only 
one examining a program that insulates public schools from 
voucher competition— finds no visible difference. No empiri- 
cal studies find that vouchers harm public schools. 

The benefits of competition in education are clearly es- 
tablished by the evidence. The only remaining question is 
whether the evidence will be permitted to shape public de- 
bate on the question of vouchers. 
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